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Abstract — THIS PAPER IS ELIGIBLE FOR THE STUDENT 
PAPER AWARD. In this paper we introduce a definition for 
nonanticipative Rate Distortion Function (RDF) on abstract al- 
phabets, and we invoke weak convergence of probability measures 
to show various of its properties, such as, existence of the optimal 
reproduction conditional distribution, compactness of the fidelity 
set, lower semicontinuity of the RDF functional, etc. Further, we 
derive the closed form expression of the optimal nonstationary 
reproduction distribution. This expression is computed recur- 
sively backward in time. Throughout the paper we point out an 
operational meaning of the nonanticipative RDF by recalling the 
coding theorem derive in 1 1 1, and we state relations to Gorbunov- 
Pinsker's nonanticipatory e— entropy |2). 

I. Introduction 

Motivated by communication and control applications in 
which lossy nonanticipative reproductions of nonstationary 
sources subject to a fidelity set are desirable, in this paper 
we introduce a definition for nonanticipative RDF. 

One envisioned application is quantization of two dimen- 
sional sources, represented by space-time processes, in which 
an IID assumption is impossed on the spatial index, when 
the time index is fixed. Such a model is considered in (T) 
to derive a coding theorem which states that the Optimal 
Performance Theoretically Achievable (OPTA) by sequential 
quantizers is given by the so-called sequential RDF, expressed 
in terms of mutual information and a conditional independence 
impossed on the fidelity set. Another envisioned application is 
source-channel matching [3|, [4|, in which nonanticipation of 
the reproduction distribution is a necessary conditions for the 
realization of this conditional distribution by encoder-channel- 
decoder maps that operate without anticipation, and hence the 
delay incurred on the end-to-end system is limited, or uncoded 
transmission Q. 

In this paper, the nonanticipative RDF is defined on abstract 
alphabets, using an information measure which is a special 
case of directed information [5] from the source sequence 
to the reproduction sequence. The main contributions are 
the treatment of the nonanticipative RDF without assuming 
stationarity of the source, the derivation of various results 
regarding existence of the optimal nonstationary reproduction 
distribution, its closed form expression via backward recur- 
sions, relations to existing coding theorems, and relations to 
the nonanticipatory e— entropy introduced by Gorbunov and 
Pinsker [2|. Since nonstationary nonanticipative RDF embeds 
the stationary nonanticipative RDF, under certain assumptions, 



our results apply for the stationary case too. Note that our re- 
sults extend the bounds developed recently in [6 1 for stationary 
Gaussian sources to general sources. 

This paper is organized as follows. Section [n]formulates the 
nonanticipative RDF on abstract spaces. Section [n]] discusses 
the relation between nonanticipative RDF with Gorbunov- 
Pinsker's nonanticipatory e-entropy and sequential RDF. Sec- 
tion JV] provides the conditions under which the existence of 
the optimal reproduction conditional distribution of nonantic- 
ipative RDF is derived. Finally, Section [V] gives the optimal 
solution for nonstationary processes. Throughout the paper we 
also include comments on the reduction of our results to the 
stationary nonanticipative RDF. 

II. Nonanticipative RDF on Abstract Spaces 

In this section, we define the information theoretic nonantici- 
pative RDF based on the methodology described in Q. 
Notation. Let N = {0, 1,2,.. .}, and N" = {0, 1,2, ... , n}. 
Introduce two sequence of spaces {(X n , B(X n )) : n G N} 
and {(y n ,B(y n )) : n G N}, where X n ,y n ,n G N, are 
Polish spaces, and B(X n ) and B(y n ) are Borel <r— algebras of 

subsets of X n and y n , respectively. Points in X N — x n£ f,jX n , 
y N = x neN y n are denoted by x = {x ,xi,...} G X N , 
y = {yoiUii • • •} € y N , respectively, while their restrictions 
to finite coordinates by x n — {xo, x%, . . . , x n } G Xo tn , 
V n = {Vo, 2/1, • • • , Vn} G yo,n, for n G N. 
Let B{X N ) = & im B(Xi) denote the cr-algebra on X N gen- 
erated by cylinder sets and similarly for B(y N ) = QigN^K^i), 
while 6(A?o, n ) and S(3^o,n) denote the a— algebras with bases 
over Ai G B{X % ), and B, G B{y z ), i = 0,1,..., n, 
respectively. Let Q(y; X) denote the set of stochastic kernels 
on y given X 0. 

A. Construction of the Measures 

Source Distribution. The source distribution 
{p n (dx n ; x n ~ 1 ) : n G N} satisfies the following conditions. 

i) For n G N, p n (-; x™" 1 ) is a probability measure on B(X n ); 

ii) For every A n G B(X n ), n G N, p„(A n ; x 71 ' 1 ) is a 
0™r o L S(Ai)-measurable function of x n ~ x G Xo n —i. 

Any distribution satisfying i), ii) is denoted by 

Pn(-;-) G Q{X n ; X , n -l). 

Let A G B(Xo tn ) be a cylinder set 



A = {x G A" N : £ G A ,xi G Ax,.. 



of the form 

G ^n}i Ai G 



B(Xi), i = 0, 1, . . . , ?i. Define a family of measures P(-) on 
B(X N ), denoted by the set Mi(X N ), given by 

P(A) = / p (dx )... Pn{dx n ]x n ~ 1 ) = /io,Ti(^4o,ri) 

(1) 

where Ao.„ = x™_ j4i. The notation Mo,n( - ) is used to 
denote the restriction of the measure P(-) on cylinder sets 
A e B(X , n ), for neE 

Reproduction Distribution. The reproduction distribution 

{q n (dy n ; y n ~ 1 , x n ) : n S N} satisfies the following 
conditions. 

iv) For n G N, g n (-|y Tl_1 , x") is a probability measure on 

B(y„y, 

v) For every B„ G B(3^„), n G N, <z„(B„; y"" 1 , x n ) is 
a ©"Tr/O^CH) © B^-measurable function of 
x n G * ,n, 6 y , n -i. 

Given a cylinder set B = {y G y n : 
yoEBo, yi&Bx, . . . , j/ n GB n }, define a family of measures 
Q(-|x) on B{y N ) by 

Q(B|x)=/ q (dy ;x )... [ q n (dy n ; y n ~\ x n ) (2) 

•'Bo JB n 

= C$ 0tn (B , n \x n ), B ,„ G B(y Q . n ). (3) 
Then Q(-|x) is a unique measure on (y N , B(3^ N )) for which 



the family of distributions {q n (dy n ;y n 



l ) : n G N} is 



obtained. 

Consider any family of measures Q(-|x) on B(y w ) satisfying 
the following consistency condition. 

CI: If D G B(y ,n), then Q(D|x) is B{X Q>n ) -measurable 
function of x G X N . 

The set of such measures is denoted by Q cl (y N , X N ). 
Then, for any family of measures Q(-|x) on (y N , B(y N )) 
satisfying consistency condition CI one can construct a col- 
lection of functions {q n (dy n ; y n ~ 1 , x n ) : n G N} satisfying 
conditions iv) and v) which are connected with Q(-|x) via 
relation (f2]l (see also Q). 

By following the methodology in [7|, given the basic measures 
P(-) on «Y N and Q(-|x) on 3^ N satisfying consistency condition 
CI, we can uniquely define the collection of conditional 
distributions {p n (-'r) G Q(X n ; <Y ,n-i) : n G N} via O, 
and {«„(■;-,■) G QCVn! ^o.n-i x <*b,n) ■ n G N} via ©, 
respectively, and vice versa, hence the distribution of the RV's 
{(Xi,Y±):ie N"} is well defined. 

Next, we introduce the information definition of nonanticipa- 
tive RDF. Given the source distribution P( ) G Mi(X N ) and 
reproduction distribution Q(-|-) G Q cl (y N ;A' N ) define the 
following measures. 

PI: The joint distribution on X N x y N defined uniquely by 

(/*0,n®#o,n)(x"=oAx£i)»^< G B(Xi), B, G BQ>i). 

P2: The marginal distribution on y K defined uniquely for Bi G 

B(y t ), i = 0,1, ...,n. by 



P3: The product distribution if o,n : B(<frj, n ) S(^o,n) ^ 
[0,1] defined uniquely for A, G B(Xi), B t G 5(3^)', i = 
0, 1, ... , n, by 

I?o.„(x" =0 (A,xB,)) = (^ ,n ® ^„)(xf =0 (AixB,)). 

The information theoretic measure of interest is a special case 
of directed information 0] defined by relative entropy B(-||-j3 

I^JX n -> F") = ©(mq,,, ^rj.JI^O.n) (4) 

log ( ° „(rf r) - JO*'" ® Qo,n){dx n ,dy n ) (5) 

= Ix"^Y"(M0,n,^0,n)- ( 6 ) 

The equivalence between and Q follows from the Radon- 
Nikodym Derivative (RND). The notation Ix"^y™(-, ■) in- 
dicates the functional dependence of J Mn (I™ — > Y n ) on 

{Mo,n) ~$o, n }- 

B. Nonanticipative RDF 

We are now ready to introduce the information definition of 
nonanticipative RDF. The distortion function do tTl (x n ,y n ) : 
Xo t n x34),ri [0, oo) is a measurable function, and the fidelity 
of reproduction is defined by 

^0,„(£>) = {3o,n G Q C1 (yo,n; *o,n) : 

^o,n(^0,n) = / d 0,n(a; n ,y n )(M0,n8)^0,n)(^" I rfy n ) 

<d\, D> 0. (7) 
The information nonanticipative RDF is defined by 

Ro,n(D) = inf Ix"->.yn(Mo )■ (8) 

If the infimum in © does not exist then ^o,n(£ ) ) = oo. 
The nonanticipative RDF rate is defined by 



1(D) = lim — — R ,n{D) 



(9) 



provided the limit exists. 

III. NONANTICIPATORY e-ENTROPY, MESSAGE 

Generation Rates and Sequential RDF 

In this section, we establish some preliminary relations be- 
tween ([8]), (O and 1) Gorbunov-Pinsker's nonanticipatory e- 
entropy and message generation rates [2 | and 2) sequential 
RDF and coding theorem |Q~). 

'Unless stated otherwise, integrals with respect to measures are over the 
spaces on which these are defined. 



A. Nonanticipatory e-Entropy and Message Generation Rates 

We recall Gorbunov-Pinsker's definition of nonanticipatory 
e-entropy [2|. Given a source Px™ G Mi(Xo^ n ) and a 
reproduction P Y , t \ x ™ G 2(3^0,™; <^b,n) introduce the fidelity 
set 



Qo,„(D) = {iV»|x»WK*) : 



— |— ( d , n {x n ,y n )P Y n\ xn {dy n \x n )®P Xn {dx n ) < d}. 
n + 1 J > 

Next we introduce two definitions from J2). 

Definition 1. (1) The source X°° = {X, : i 6 N} is 

called "specified" if P Y ^\x k G Qo,k{D) and P Y ™ ^xj? x S 
Qk+i.n(D) implies P Y k t y^ +1 \x k .x% +1 £ Qo.n(D), and it is 
called "consistent" if the reverse holds. 

(2) 77;e source is called stationary if {Xi : i G N} is 
stationary and for any k, Qo t n{D) and Qk, n +k{D) ore copies 
of the same set. 

Typical example are stationary sources with single letter 
fidelities d , n (x n , y n ) = Y*=a P( x i> Vi)- For a specified 
source, Gorbunov and Pinsker restricted the set Qo,n(-D) to 
those reproduction distributions which satisfy the Markov 
chain (MC) X™ +1 O X n o Y n ^ P Yn{x ^ (dy n \x°°) 



Y"\X 



..{dy n \x n ) 



a.s., n 



0,1,..., and introduced the 



nonanticipatory e-entropy defined by 

A 



inf 

P Y n lx neQ ,„(D>- 



I(X n ;Y n ). (10) 



If the infimum in ( fTOb does not exist then R 7 Q a n (D) = oo. 
The difference between the classical RDF [9| and (TTOb is the 
presence of the MC, which implies that for each i, Yi is a 
function of the past and present source symbols X 1 , and it is 
independent of the future source symbols X" +1 , nel 
Moreover, the authors in [2 1 also introduced the nonanticipa- 
tory message generation rate of the source defined by 

R na (D)= lim -L-B™(D) (11) 

n— >oo n + 1 

provided the limit exists. 

An alternative definition of the nonanticipatory message gen- 
eration rate of the source is defined by (]2] 

(D) = inf lim IiyX ,Y '- (12) 

P Y <x>\x<*>£Qo,aa(Dy. n^-OO U+l 

X°^ 1 ^X i ^Y i , i=0,l,... 



R 



na,+ 



whenever the limit exists. Clearly, in general R na ' + (D) > 
R na {D). 

The main results derived in (2] are the following. 

GP 1: If the source is stationary for some finite k, i.e., 
R o°k( D ) < d = constant < oo, then R na {D) is 
defined and it is finite, and R na (D) > R na '+(D), 
hence R na (D) = R na -+{D). 

GP 2: If the source is stationary and consistent then 
R na (D) is always defined, and either 1) for 
some finite k, R[] a k {D) < oo which implies 



lim^oo T±jR% a k {D) = R na {D) < oo, or 2) for 
any k G N, Rq%(D) = R na (D) = oo. 
GP 3: For a single letter distortion and stationary source, 
if R na (D) = R na >+(D) then the analysis of in- 
fimum in (fT2l is realizable in terms of stationary 
source -reproduction pairs {(Xi,Yi) : i G N} 
such that P Yn \x°°(dy n \x°°) = P Y n\ X n{dy n \x n ) 
and Py-| X ~ ® P x - G Qa, n (D). 
With respect to Gorbunov-Pinsker's definition of nonanticipa- 
tory e-entropy and message generation rate, 1) we show that 
([Tol l reduces to the information definition of nonanticipative 
RDF ([8), 2) under certain general conditions the infimum in 
([Tol l exists and it is finite, and, 3) we derive an expression 
of the optimal reproduction distribution which achieves the 
infimum in © or (TlOb . 

First, we establish the connection between nonanticipatory 
e-entropy ( fTOb and nonanticipative RDF ([SJ by utilizing the 
following general equivalent statements of MCs. 

Lemma 1. The following are equivalent for each n G N. 

(1) P Y « {xn (dy n \x n ) = t Ynlxn (dy n \x n )-a.s.; 

(2) For each i = 0,l,...,n-l, Y, <-> (X\Y l - x ) o 
(Xi+i, X i+2 , ...,X n ), forms a MC; 

(3) For each i = 0, 1, . . . , n — 1, Y l <H» X 1 <H> X l+ i forms 
a MC; 

(4) For each i = 0, 1, . . . , n — 1, O X* O F\ /orms 
a MC. 

Proof: The derivation is straightforward. ■ 
By utilizing Lemma [1] then R 0>n (D) = Ro a n (D). Then the 
extremum of the nonanticipatory e-entropy ( fTOb is equivalent 
to the extremum of i^o.n(-D) given by ®. 

B. Sequential RDF and Coding Theorems 

— > 

Next, we establish a coding theorem for i?o,n(-D) using the 
information sequential RDF introduced by Tatikonda in (TJ, 
which utilizes a similar formulation to the nonanticipatory 
e-entropy. The coding theorem is derived by considering a 



two dimensional source X r 



A 



0,...,n,j 



0, . . . , s} G ®" = o ®j=o %i,3' where * represents time index 
and j represents spatial index. The coding theorem is based 
on the following definitions. 

Definition 2. A sequential quantizer is a sequence of mea- 
surable functions /" = {fi : i — 0,1,..., n} defined by 

fi : x^ s x y*-^ s i ^ y?, y? = f{x^, y ^ s ), i = i,..,,n. 

The set of all such quantizers is denoted by J- n ,s . 
Definition 3. Let Qq^ a '°{D) denote the fidelity set 

i - 

i=0 

where p s : X? x y? i— > [0, oo) : i = 0, 1, . . . , n is measurable. 
The operational sequential RDF is defined by 

1 

" / / ) \ _ i . , I 



inf 



/»eQf„ D ;»(D) (n + !)(« + !) 



i?(y s ,...,y„ s ) 



and the operational sequential RDF rate is defined by 



R 



SRD.o 



(D) = lim R™ D S >°(D) 



The information sequential RDF for which a coding theo- 
rem is derived in Jl] is the following. Given the two di- 
mensional source Px^,= (dx n,s ), a reproduction distribution 

P Y n,s\x™,s{dy n ' s \x n ' s ), and a fidelity set 

— ^E^ y ,p s K,7/) < D} (13) 



i=0 



the information sequential RDF is defined by 

R S o% D s(D) = inf 



(Xr , +1 ,...,x=)«-(x i ' s ,y i " 1 - s )«y i s , i=o,i, 



{ 



1 



-I(X n < s ;Y n < ! - 



>}■ 



(14) 



(n+l)(s+l) 
The sequential source coding theorem is the following. 

Theorem 1. (Sequential Source Coding Theorem [1 ]) Suppose 
{Xi j : i = 0,1, ... ,n,j = 0,1, ... ,s} are finite alphabets, 
P x is{dx n ' s ) = ®» j=0 P{dx]), {X™ : j = 0,1,..., s} 
identically distributed, and there exists an xq and D max > 
such that Ep x p s (Xij, xq) < D max , for all i = 0, 1, . . . , n, 
j = 0, 1, . . . , s. Then for any e > and finite n G N, there 
exists s(e,n) such that for all s > s(e,n) we have 

?SRD, 



where 



R S o^°(D + e)<R^(D)+e 



inf 



I(X n ;Y r ' 



D y«|xn: w i T E Px „ yn {i:*' =0 p(X < ,r i )<£'} 

x i+l<->( xi ,Y i ~ 1 )^Y i : i i;. I n 



(15) 



Notice that Rq^ d (D) is precisely Gorbunov-Pinsker's nonan- 
ticipatory e-entropy -^p^R r Q a n (D) Q. Moreover, by Lemma Q] 

o,n(D) given by ©. However, the coding 
Theorem [T] is valid for finite time index n, since its derivation 
is based on taking s — > oo. Consequently, with respect 
to the information sequential RDF (fT5T l. our objective is to 
find the expression of the nonstationary optimal reproduction 
distribution. 

IV. Existence of Reproduction Conditional 
Distribution of Nonanticipative RDF 

In this section, the existence of the minimizing (n + l)-fold 
convolution of conditional distributions in ([8} is established 
by using the topology of weak convergence of probability 
measures on Polish spaces. In fact, our results are more general 
than what is envisioned by the assumptions in Q~), 0, since 
we work with abstract Polish spaces and general distortion 
functions. First, we recall some properties from fl7]. 



Theorem 2. [7] Let {X n : n G N} and {y„ : n e N} be 
Polish spaces. Then 



(1) 

(2) 



(3) 



The set Q cl (3 ; o n> <^b,n) is convex. 

o, n ) is a convex functional of 
Qo,n G Q ci {y . n ; Xo,n) far a fixed p 0in G 

Ml(Xo,n). 

The set Qo,n(D) is convex. 



Let BC(yQ tn ) denotes the set of bounded continuous real- 
valued functions on 34), n- Below, we introduce the main 
conditions for the existence of nonanticipative RDF ([SJ. 

Assumption 1. The following are assumed. 

(Al) 3b, n is a compact Polish space, Xo_ n is a Polish 
space; 

(A2) For all h(-)^BC(yo_ n ), the function 
mapping (x n ,y n ^ 1 ) G X ,n x y , n -i ^ 
Jy h(y)q n (dy]y n ~' 1 ,x n ) G R is continuous 
jointly in the variables (x n , y n ) G X®, n x 3 , o,n-i/ 

(A3) do iTl (x n ,-) : 3b, n [0, oo) is continuous on 3'o,n> 
uniformly in x n G Xq^; 

(A4) There exist sequence (x n ,y n ) G Xq u x 3b, n satisfy- 
ing do, n (x n ,y n ) < D. 

The following weak compactness result can be obtained, which 
will be used to establish existence. 

Lemma 2. Suppose Assumption\J](Al), (A2) hold. Then 

(1) The set Q (3fo,n> Xo,n) is weakly compact. 

(2) Under the additional conditions (A3), (A4) the set 
Qo,n(D) is a closed subset of Q ci (3b,ni Xo, n ) 
(hence compact). 

The next theorem establishes existence of the minimizing 
reproduction distribution for (|8). First, we need the following 
Lemma. 

Lemma 3. [71 Under Assumptions [7] (Al), (A2), 
Ix«->Y" {lM),m Qo,n) is lower semicontinuous on 
Qo,n G Q C1 Xo,n) .far a fixed sMi(Xo, n ). 
Next, we state the main Theorem. 

Theorem 3. (Existence jj^j) Suppose Assumption 1 hold. Then 
Ro,n(D) nas a minimum. 

Thus, Theorem [3] implies the following results. By GP1, 
for a stationary source R na (D) — R na ' + (D) is defined 
and it is finite, by GP2, for a stationary consistent source 
limfc^oo j^RqUD) = R na (D) < oo, and by GP3, for 
a stationary source and single letter distortion, R na (D) = 
R na ' + (D) and the infimum in (TTZb is realized by stationary 
source-reproduction pairs. 

V. Optimal Reproduction of Nonanticipative RDF 

In this section, we derive the expression of reproduction con- 
ditional distribution which achieves the infimum of Rn. n (D) 
or Rq^ d (D). We assume a distortion function of the form 



do. n {x n ,y n ) = 2~2i=o Po.i{x l , y 1 )- We shall need, the Gateaux 



differential of lx n -¥Y n (p>o,n> Qo n ) m everv direction of 
{li(dyi) x 1 ) : i = 0, 1, . . . , n} (due to nonstationarity). 

Theorem 4. (Gateaux Derivative) Let l^ l0n (qi : i = 
0,1,..., n) = Ix'^Y™ (Mo.n, <^o,n) be well defined for 
every ^ ,« S Q cl (3 ; o,n; ^o,n)- T^en {%(•;•,•) : i = 
0,1,..., n} — > „(3i( - ; ■, ■) : * = 0,1,..., n) w Gateaux 
differentiate at every point in Q(3^; 3^0, i-i x ^b,i)> 
Gateaux derivative at the points q* {■]■,■) in each direction 
Hi = Qi - Q*- i = 0, ■ • ■ , n, is 

ffl-no.nfai'Qi-Qi -i = 0,...,n) 



A 



For k = 0, 1, . . . , n, i = n — k, we get 



Qn(dy n ]y n 1 ,x n ) 



— (17) 

f yn e s P°.»(* n >y n )v*{dy n ;y n - 1 ) 

J y e s P°^ i ^ i )-9i,n^ i ,y i )ul(dy i ;y i - 1 ) ' 

(18) 



The nonanticipative RDF is given by 



i=0 



E / 1°: 



q*{dyi;y % d 



vfidy^yi- 1 ) J de 



l , n (D) = s(n + l)D-J2 1(1 g l ,nq*(dy t ;y l -\x l )+ 
X^(<%V) | e=o M 0)i (^) log y e^*-^^*^;^- 1 )) x ^S,»-i (^V* -1 )a*o,« C^) 



where Q e oi = ®) = r l q j {dy j ;y j 1 ,x j ), q] = q* + e(qj - <?*), 
j = 0, 1, i = 0, 1, ...,n, 

Sq (dy ;x ) 



j^o(dyo;x ) 



e=0 



e=0 



= 3qo(dyo;x ) ® qKdy^yojX 1 ) 
+ qo(dyo;x ) <g) 5q 1 (dy 1 ;y ,x 1 ) 



-j-cfo.iOfoV) = Sq (dy ;x°) ®} =1 q^dy^f- 1 ,x j ) 
+ ^oC^ojiCQj^iCdj/ilyca; 1 ) ®}=2 qj(dyj;y J ~ l ,x J ) 

Proof: The derivation is lengthy, hence it is omitted. ■ 
The constrained problem defined by © can be reformulated 
using Lagrange multipliers (due to its convexity) by utilizing 
Lagrange Duality Theorem ifTUl to obtain 

Ro,n(D) = inf Ix«->-y™(Mo,n> Qo,n) 
Qo,«ea ,„(D) 



max 

s<0 



inf 



(^.»(^o,«)-(n + l)^)} 



!o,»eQo,n(D) 



where s s (oo, 0] is the Lagrange multiplier. 
Since q i (dy;y' l ~ 1 ,x l ) e Q(X;iMM X A? ,i), one should in- 
troduce another Lagrange multiplier to obtain an optimization 
problem without constraints. This process is involved; hence 
we state the final results. 

General Recursions of Optimal Non-stationary Reproduc- 
tion Distribution. The general recursions are the following. 
Define 



g n Ax n ,y n ) = o 
g n - k , n (x n -\y n - k 

log I / - "->=+iO 



Pn-k+l(dx n -k+V,X n k ) 



*<-k+i{dy n -k+i;y n k )), k=l,...,n. (16) 



Discussion. The above recursions illustrate the nonanticipa- 
tion, since gi tn (x l , y l ), i = n — k, k = Q, 1, . . . , n, appearing 
in the exponent of the reproduction distribution (fT8l integrate 
out future reproduction distributions. Note also that for the 
stationary case all reproduction conditional distributions are 
the same and hence, gi, n {-,-) = 0, which implies q* n (•;•,•) 
is given by ( TTTb . The above recursions are general, while 
depending on the assumptions imposed on the distortion 
function and source they can be simplified considerably. 

VI. Conclusion 

In this paper, we derive an analytical closed form expression 
for the nonanticipative RDF for nonstationary processes, and 
we relate the definition of nonanticipative RDF to other works 
in the literature. 
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